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Abstract 

This article reports on findings from a multiple case 
study investigating the nature of educators' approaches to¬ 
ward monitoring English language learners' (ELLs) perfor¬ 
mance and using data to improve instruction and apply ap¬ 
propriate interventions. Six New York elementary schools 
where ELLs 1 performance was better than predicted (i.e. 
odds-beating) based on student assessment data were stud¬ 
ied. The analysis revealed that several strategies were com¬ 
mon among the schools studied and were associated with 
the schools' better ELL performance outcomes. These in¬ 
clude: 1) connecting instruction and interventions to "real 
time" data based on multiple measures of student perfor¬ 
mance including benchmark and formative assessments; 
2) communicating performance via technology among teach¬ 
ers and with family members and legal guardians; 3) col¬ 
laborating through routines among teaching and support 
staff as well as school and district leaders. Implications for 
district and school leaders and teachers are discussed. 
Implications for district and school leaders as well as teach¬ 
ers and other instructional specialists are offered. 

How Odds-beating Elementary School Educators Monitor 
and Use Student Performance Data 

New York is one of the top 15 states with the high¬ 
est density of English language learners (ELLs) in its schools 
(Ruiz Soto, Hooker & Batalova, 2015); in the 2015-16 school 
year more than 200,000 students comprising over 8% of the 
total school population were identified as ELLs 1 . This num¬ 
ber has increased 20% over the past ten years (New York 
State Education Department, n.d.) and reflects a nationwide 
trend toward more linguistically diverse classrooms (U.S. 
Department of Education, n.d.). 

This study presents data which was gleaned in a 
larger study conducted in the 2015-16 school year in six 
odds-beating elementary schools identified for ELLs' above- 
predicted performance. The researchers focus on ways edu¬ 
cators in odds-beating schools approached monitoring and 
using ELL's performance data since this was found to be an 
important factor related to better outcomes. Other findings 


and full case studies of the schools are available on the 
project website (see, Authors, n.d.). 

While some ELLs arrive in school well-prepared 
to succeed, others have a variety of needs, social/emo¬ 
tional, physical, academic, and otherwise, that need close 
monitoring and appropriate and timely responses by teach¬ 
ers, as well as specialists and school and district leaders. 
By examining "odds-beating" schools (i.e. those with rela¬ 
tively better ELL performance outcomes taking into account 
school demographics such as poverty and diversity), this 
study sought to identify promising practices that support ELLs' 
success in school. 

Related Literature 

For this study the researchers briefly discuss the 
monitoring and use of data to improve instruction and inter¬ 
ventions for ELLs. Performance data for ELLs falls into three 
main categories: language proficiency assessments, con¬ 
tent knowledge assessments, and classroom-level assess¬ 
ments. Bailey and Carroll (2015) argue that "macro-level as¬ 
sessments," (i.e. standardized assessments that measure 
English language proficiency or content knowledge) are not 
meant to be used for instructional purposes and are inad¬ 
equate for guiding instructional decisions. Rather, school¬ 
wide and district-wide processes and practices that system¬ 
atize teachers', specialists’, and school and district leaders' 
use of formative data are associated with better student out¬ 
comes for all students (Authors, 2013: Stosich, 2016). 

While macro-level assessments are necessary 
and fulfill particular needs for inter-school comparisons 
and trend analyses, research studies have indicated that 
formative assessment data is essential in facilitating teach¬ 
ers' use of effective instructional strategies and application 
of appropriate interventions for ELLs. As a number of schol¬ 
ars have expressed, (Abedi, 2010; Heritage, Walqui & 
Linquanti, 2013, Bailey & Carroll, 2015; Duran, 2008, Heri¬ 
tage & Heritage, 2011) frequent formative assessment are 
particularly important for ELL students as many undergo 


'see https://data.nysed.gov/enrollment.php?year=2016&state=yes 
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significant change in their language competencies as com¬ 
pared to their monolingual peers in relatively short periods 
of time. Younger ELLs, in particular, may experience the 
language learning and socialization process especially 
quickly, requiring teachers to assess and adjust instruc¬ 
tion frequently. In a study of four districts, Hakuta (2000) 
and her colleagues found that ELL students' oral proficiency 
improved much faster from grade one to grade three, than 
from grade three to grade five. A number of studies sug¬ 
gest that in general elementary-aged ELLs 1 learning pat¬ 
terns require teachers to pay close attention to their devel¬ 
opment by providing instruction based on constant moni¬ 
toring of their performance (Hawkins, 2004, 2005; Oga- 
Baldwin & Nakata, 2014). 

For formative assessment to be useful, research 
shows that several factors need to be considered. First, 
formative assessments, like all assessments, should be 
free of linguistic and cultural biases to provide an accurate 
measure of ELL's competencies and knowledge (Abedi, 
2010). Second, English Second Language (ESL)/English 
New Language (ENL) teachers as well as mainstream class¬ 
room teachers and specialists need to understand how to 
create and use such formative assessments as well as 


provide feedback to ELLs that focuses on content and qual¬ 
ity versus an over-emphasis on "surface" level language 
competencies (i.e. those that do not affect understanding of 
the intended message) (Alvarez, Ananda, Walqui, Sato, & 
Rabinowitz, 2014; Bailey & Carroll, 2015). 

Research Methods 

In this multiple case study we utilized a replicated 
"unusual case" design to identify patterns in schools char¬ 
acterized by relatively better elementary ELL outcomes 
(Yin, 2014, p.57). Quantitative methods, specifically, regres¬ 
sion analyses, were used to identify the sample. 

Sample Selection 

Schools were identified based on performance 
outcomes as well as a variety of other demographic crite¬ 
ria. Performance outcome measures included the 2012- 
13 and 2013-14 New York State Mathematics and English 
Language Arts assessments across grade levels. Schools 
classified as "odds beating" are ones in which ELLs ex¬ 
ceeded expected average performance in ELA and math¬ 
ematics at grade 3 through grades 5 or 6 on the two state 


Table 1. 

Demographics of the Odds-Beating Schools 2 

School 

Total 

Enrollment 

Urbanicity 

% ELL 

% Economically 

Disadvantaged 3 

PPE 4 

Catskill 

760 

Rural 

3 

62 

$24,032 

Schuylerville 

718 

Rural 

2 

31 

$17,884 

Fostertown 

637 

Urban 

11 

61 

$21,878 

Van Rensselaer 

622 

Urban 

8 

73 

$19,870 

Guilderland 

548 

Suburban 

11 

18 

$17,995 

Blue Creek 

482 

Suburban 

6 

24 

$18,457 

NYS average 

N/A 

N/A 

8 

54 

$21,812 



2 Data are from the 2014-15 State report cards. 

3 One measure of poverty, and the one used here, is economic disadvantage (see definition at 
https://data.nysed.gov/glossary.php?report=reportcards). 

38 “2013-14 districtwide total expenditures per pupil. 







assessments. Using Statistical Package for the Social Sci¬ 
ences (SPSS) software, an expected average performance 
level was generated for each subject at each grade level. By 
comparing expected to actual average performance, schools 
could then be classified as "odds beating" if the difference 
between expected and actual performance was close to one 
standard deviation greater than the mean difference for all 
schools in the state. Of 1,378 schools serving grade 3 
through grades 5 or 6 outside of New York City, 127 schools 
were identified as potential "odds-beaters." 

The sample was then investigated to identify only 
those schools in "good academic standing" for ELL per¬ 
formance and further winnowed into three categories: 
rural, suburban, and urban schools and those serving 
more or less economically disadvantaged and/or ethni¬ 
cally diverse student populations, favoring both higher 
poverty and higher diversity in the final sample to meet 
the criteria of "odds-beating." Finally, schools whose per- 
pupil expenditures, combined wealth ratio, and percent¬ 
ages of expenditures on instruction were above the norm 
were eliminated from the sample such that only schools 
with average to above average demographic challenges, 
yet higher than predicted performance would be studied 
(see Table 1). 

It is important to note that the schools in this study 
are not the highest performing with regard to their ELLs' per¬ 
formance in NY state, but rather defy the typical performance 
pattern taking into account poverty and diversity. As displayed 
on the histogram in Figure 1, the case study schools are 


above the mean for ELLs' performance in comparison to 
other schools in the state taking into account school demo¬ 
graphics such as poverty and diversity. 

Data Collection 

Once schools were chosen and site visits arranged, 
the research team, trained in using the instruments and in 
human subjects research, visited each school to collect docu¬ 
mentary evidence and conduct interviews with the focal stu¬ 
dents, teachers, principals and other building leaders, and 
district administrators. Also, we also conducted focus groups 
with teachers and conducted classroom observations. In¬ 
terviews and focus groups, which were audiotaped and then 
transcribed, followed a semi-structured protocol guided by 
the research questions. Documentary evidence included les¬ 
son plans, student work, curriculum maps, and other in¬ 
structional materials. Class observations were guided by 
an observation protocol that prompted description of teach¬ 
ing and learning activities as well as a brief interview with 
teachers about the intent of their lessons. Before and during 
site visits researchers kept interpretive memos cataloguing 
questions raised, notes for follow up, and beginnings of 
interpretations. 

In total, 25 administrators, 41 teachers, 7 support 
staff, and 17 student focus group or interview transcripts, 
as well as 28 classroom observation notes were collected 
and coded inductively using a constant-comparison 
method utilizing the qualitative software program 
HyperResearch (Strauss & Corbin, 2008). Using typical 
cross-case procedures, code reports by 
theme related to the research questions 
were produced and a matrix comparing 
themes was utilized to identify key patterns 
among the schools (Yin, 2014). The re¬ 
search team then engaged in axial coding 
in order to chunk major themes and their 
relationships as they related to the re¬ 
search questions (Strauss & Corbin, 2008). 

This was facilitated again through the use 
of HyperResearch and data matrices in 
Excel (Miles, Huberman, & Saldana, 2014). 

Findings 

“You can't get somewhere if you don't know 
where you're going.” - Fostertown ETC prin¬ 
cipal 

This study was foregrounded in lit¬ 
erature that highlights the relationships be¬ 
tween effective monitoring and use of ELL 
performance data and ELL performance 
outcomes. Through the analysis of inter¬ 
view, focus group, observation, and docu¬ 
mentary evidence data the researchers 
identified a combination of performance 
monitoring processes and practices common 
among the odds-beating schools studied. 39 



Figure 1. Odds-beating school ELLs’ performance on ELA and 
Mathematics state assessments in 2013 and 2014 as compared 
to other schools in New York. 
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Three characteristics of ELL progress monitoring in odds- cuss student performance and work together to develop les- 
beating schools were identified: (1) connecting instruction sons and materials to address the gaps in learning as dis¬ 
and interventions to "real time" data based on multiple mea- played by the data. Response to Intervention (RTI) is also 
sures of student performance including benchmark and for- linked to this collaborative data analysis. When Tier 1 inter- 
mative assessments; (2) communicating performance via ventions are not producing any change, "real-time" class- 
technology; and (3) collaborating on instructional and other room data informs teachers' next steps in referring students 
interventions ELLs' need through routines among teaching to a Tier 2 intervention, 
and support staff as well as school leaders. 

Educators at Fostertown ETC and other odds-beat- 
Connecting Instruction and Interventions to "real time" data j n g elementary schools analyze data for causes of students' 

poor performance and seek solutions. In Blue Creek ES, for 
In schools with odds-beating ELL performance, example, a district leader described how the principal is 
teachers and leaders pointed to the weaknesses of relying "constantly looking and using her faculty meetings for [shar- 
upon state assessment and the New York State English as j ng data analysis], 'Okay, here's a problem that I identified 
a Second Language Achievement Test (NYSESLAT) assess- because of the data. What are some things that we can try?' 
ment data to inform instruction and interventions for their For example, the principal noted gaps in attendance at open 
ELL students, as they do not provide actionable information houses and parent teacher conferences and then engaged 
needed to help their students grow and learn in real time. j n an intervention. A district leader described the principal's 
Instead, they reported relying upon frequent formative as- approach: 
sessments, benchmark assessments and other data to in¬ 
form them. Figure 2 shows Fostertown ETC's "Data Analy- She [the principal] was looking at who is coming 

sis and Planning Sheet," which includes a list of seven dif- to open-house night, who's coming to confer- 

ferent types of assessments that inform teachers about ar- ences, who's not, and realizing which subpopu- 

eas of concern, used to generate goals and specific teacher lations and which families weren't able to get to 

actions to meet those goals. Fostertown ETC's educators those. So she worked with the transportation di¬ 
meet in Professional Learning Communities (PLCs) to dis- rector to get a school bus to go to some of these 



populated apartment complexes and bring the 
people, because they didn't necessarily have a 
way to get a ride to school for these things. 

Complementing these strategies, Blue Creek 
teachers also gather a variety of benchmark and formative 
assessments, the same assessments as their non-ELL 
peers, with reading benchmarks administered in Septem¬ 
ber, January, and May. These are used to identify, "sight word 
growth, letter sounds, knowledge of letters, and . .. progress 
with reading," according to a classroom teacher. In math, 
Blue Creek ELLs take chapter tests and a middle and end- 
of-year computerized assessment, at which time they are 
offered the accommodation of question read alouds. Alto¬ 
gether, these formative assessments provide teachers with 
a variety of information about ELLs 1 growth in language and 
content and allow educators to intervene in an appropriate 
and timely fashion so that each learner can make progress. 

Collaborating through routines among teaching and sup¬ 
port staff 

Importantly, leaders in the odds-beating schools and 
districts valued teachers' desires to collaborate around in¬ 
struction, assessment, and intervention for ELLs. As in the 
other schools studied, Fostertown ETC's educators benefit 
from a variety of opportunities to collaborate. Scheduled 
grade-level meetings are used to analyze students' perfor¬ 
mance data. In these meetings, principals, teachers, and 
specialists focus on classroom-level assessments as they 
believe them to be more meaningful for guiding instruction. 
A school leader explained how this is done: 

We went into using data protocols during meet¬ 
ings to really guide the meetings. We only have 
thirty-five minutes, so we need to be laser fo¬ 
cused. So we started using data protocols and 
we're doing student work. A lot of the discussion 
I had with teachers when I first came in is, "How 
do you use data? How do you use data in your 
classroom, as a grade level or per building? 

And we have the over-arching New York State 
data assessments and those achievements, 
but what is real-time data? State data just tell 
us at the end of the year if you have mastered 
the state-level standards. But what is going on 
in your classroom in real time?" So we put pro¬ 
tocols in place as far as reviewing student work. 

So the teachers bring in either exit tickets that 
they've done with their kids or any kind of work 
or problem sets they want to go over. 

In addition, Fostertown ETC educators benefit from 
collaborative processes and practices centered on progress 
monitoring at the district level. The Superintendent initiated 
the practice of "DataCon" or "Data Conversations" in which 
all building and district leaders meet to discuss the results 
of local and state assessment data for each school. At this 
meeting, each principal is called to the table to discuss a 
variety of data points. All of the cabinet members and central 


office administrators as well as all building principals are 
present and encouraged to ask questions of each principal 
in "the hot seat." A district leader described it as "a fishbowl 
conversation" that is "both evaluative and supportive in na¬ 
ture," as it often inspires new ideas for principals to try in 
their own buildings. This practice encourages consistency 
across the large urban district and holds building and dis¬ 
trict leaders accountable for making the best use of multiple 
measures of student performance. 

Likewise in suburban Blue Creek Elementary, 
teachers provide "benchmark profile sheets" to the principal 
and upload their letter grades to the online portal that allows 
for the sharing of data district-wide. A reading specialist ex¬ 
plained how classroom and ENL teachers collaborate by 
sharing their assessment data: 

Usually once we do assessments, we record 
all of our students on a class form and return it 
to the teacher, and usually at some point we 
end up touching base with the ENL teacher in 
person or through email to let her know about 
student progress. 

In the smaller schools studied, some with only one 
ENL teacher in the school building, collaboration is less 
formal, but nonetheless recognized as essential to meet 
ELLs' learning needs. For example, because there are not 
many ELLs in the school and one ENL teacher in 
Schuylerville, the ENL teacher monitors the progress of ELLs' 
language development most closely. Yet, classroom teach¬ 
ers and ENL teachers say they keep each other informed of 
areas in need of more attention consistently. According to a 
Schuylerville teacher, 

We (classroom teachers) are monitoring all the 
time, whether it's regular assessments on tests 
or things like that. We're always doing informal 
assessments, just walking around and check¬ 
ing, and then I'm assessing them by collaborat¬ 
ing with the service providers, the ENL, and the 
special ed teacher. 

Through both formal and informal processes and 
practices that encourage collaboration around progress 
monitoring educators work synergistically to meet ELLs' 
academic needs. 


Communication among mainstream classroom 
teachers, ENL teachers, special education teachers, and 
AIS teachers via technologies allow for communications to 
occur in a timely manner. Such data management systems 
as "Infinite Campus" or "School Tool" allow educators to 
share formative, benchmark, and assessment data with 
one another. For example, Blue Creek teachers use their 
school management system, "Infinite Campus," to com¬ 
municate student performance on benchmark exams with 
the school principal. In addition, all grades and other data 41 
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(e.g. attendance) are stored there so that educators can 
easily share data with one another. 

Similarly, in other schools like Guilderland ES, 
School Tool was identified as facilitating district-wide data 
analysis of ELLs' performance and also as a way to com¬ 
municate with family members and legal guardians. 
Through an information night, Guilderland parents are in¬ 
vited to the school to learn how to use features of School 
Tool such as accessing their children's grades and state 
test scores, as well as other important school information. 
According to a teacher, the school district has worked hard 
at "communicating with parents to [use the program]. The 
ability to disseminate student data to families of ELLs was 
described as beneficial in developing a positive and open 
relationship between families and the school. Because 
"many families [of ELLs] believe very highly in education and 
they're very strong on their students," one school leader ex¬ 
plained, "School Tool provided an effective avenue to learn 
about their children's progress without having to come to 
school or schedule a meeting with teachers". 


Figure 3. Blue Creek apps 
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In addition, instructional technologies such as 
clickers, Chromebooks, and apps (see Figure 3 for ex¬ 
ample of app offerings in Blue Creek ES), were reported to 
be used frequently to provide immediate feedback to stu¬ 
dents and formative assessment data to teachers. These 
data inform teachers as to the strengths and weaknesses 
of students so that differentiated instructional strategies 
can be planned as a next step. 

Systematizing the communication of ELL student 
performance via technology allows principals, specialists, 
classroom teachers, and ENL teachers to provide a consis¬ 
tent and informed program of instruction for ELLs. In addi¬ 
tion, sharing data with parents and guardians encourages 
families and legal guardians to participate actively in the 
education of their children and establishes a connection 
between home and school. 

Recommendations 

Although this study is limited to data culled in only 
six schools, and is thus, not generalizable to all schools in 
the state of New York, it suggests a few considerations for 
practice related to data monitoring and use, namely: 

> Provide systems to carefully monitor the progress 
of ELLs using multiple measures- both formative 
and summative, connected to systems of interven¬ 
tions for students who need more support. 

> Provide systematic progress reporting and com¬ 
munication/dissemination routines for all educa¬ 
tors and specialists who interact with ELLs as well 
as their families 

> Ensure that adequate ENL expertise is within each 
school building and that collaboration among ENL 
teachers, mainstream classroom teachers, and 
specialists across the school and district can oc¬ 
cur on a regular basis. 

Conclusion 

While the number of ELLs in New York schools and 
in other states around the nation continues to increase, trou¬ 
bling trends of ELLs falling behind as they progress from 
elementary to secondary school persist (U.S. Department 
of Education, n.d.). Indeed, children's experiences of suc¬ 
cess in elementary school have been found to strongly cor¬ 
relate to their trajectories in secondary school and beyond 
(Akiba, LeTendre, & Scribner, 2007; Suarez-Orozco, Gaytan, 
Bang, Pakes, O'Connor, & Rhodes, 2010). 

In some New York elementary schools, ELLs are 
beating the odds with better than expected performance out¬ 
comes. The findings from this study demonstrate that what 
educators do to monitor and use performance data can re¬ 
late to better performance outcomes. 
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